REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson 

Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 

1 .  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

June  1997  Tech  Report 

4.  TITLE  AND  SUBTITLE 

Approximation  Probability  of  Detection  in  the  Janus  Model 

5.  FUNDING  NUMBERS 

6.  AUTHOR(S) 

Mickey  A.  Sanzotta 

E.  Todd  Sherrill 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

OPERATIONS  RESEARCH  CENTER 

USMA 

WEST  POINT,  NY  10996 

10. SPONSORING  /  MONITORING 

AGENCY  REPORT  NUMBER 

1 1 .  SUPPLEMENTARY  NOTES 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 

DISTRIBUTION  STATEMENT  A: 

APPROVED  FOR  PUBLIC  RELEASE:  DISTRIBUTION  UNLIMITED 

19 

12b.  DISTRIBUTION  CODE 

980804  063 

13.  ABSTRACT  (Maximum  200  words ) 

This  report  documents  our  approach  at  developing  a  satisfactory  detection  model  for  use  in  computing  imw^ _ j. 

the  Janus  Wargame.  Information  gain  measures  the  Blue  forces'  awareness  of  Red's  disposition  over  time.  Within  a  time 
interval  (t,  t+1)  the  measure  is  a  distance  measure  between  two  probability  distributions  Pt  and  Pt+1  respectively.  These 
distributions  represent  the  relative  discrete  probability,  from  Blue's  perspective,  that  a  Red  vehicle  is  in  a  particular  area  of 
the  battlefield.  The  sum  of  the  discrete  probability  values  over  all  areas  is  1.0  with  those  areas  of  greatest  likelihood  having 
the  larger  values.  Information  is  generated  by  the  actions  of  Blue  sensors.  When  any  Blue  sensor  scans  an  area  of  the 
battlefield  Blue  gains  information  about  the  enemy  disposition.  The  magnitude  of  this  new  information  is  determined  in  part 
by  the  efficiency  of  the  Blue  sensor.  As  new  information  about  the  location  of  a  Red  vehicle  becomes  available  to  Blue  we 
update  the  probability  distribution  using  theoretical  formulations  based  upon  Bayes  formula  [4] . 

Though  the  theory  is  simple,  its  implementation  in  the  Janus  model  was  very  challenging.  The  Bayesian  formulation 
mentioned  above  requires  two  parameters  during  each  time  stage:  1)  a  listing  of  which  areas  of  the  battlefield  (cells)  Blue 
sensors  looked  in  and  2)  the  probability  of  detection  (PD)  for  the  sensors  that  did  the  respective  scanning.  Neither  of  these 
data  are  directly  available  in  Janus  runs,  nor  can  they  be  deduced  from  Janus  output  files.  For  example,  the  Janus 
algorithms  for  line  of  sight  computations  and  detection  of  enemy  vehicles  are  only  called  when  two  opposing  vehicles  are 
within  some  threshold  of  proximity  to  each  other.  Since  information  gain  credits  finding  where  the  enemy  is  not,  we  need  to 
know  at  each  time  increment  what  terrain  cells  Blue  sensors  have  searched  regardless  of  the  presence  or  absence  of  enemy 
vehicles.  Likewise,  we  need  to  know  what  the  PD  would  have  been  for  each  particular  sensor  /  cell  pair  had  there  been  an 
enemy  vehicle  present  when  the  sensor  searched  the  cell.  For  our  purposes,  a  sensor  is  considered  to  have  searched  a  cell  if 
it  has  unobscured  line  of  sight  between  its  position  and  the  particular  terrain  cell. 

14.  SUBJECT  TERMS 

JANUS 

15.  NUMBER  OF  PAGES 

20 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION  18.  SECURITY  CLASSIFICATION  19.  SECURITY  CLASSIFICATION 
OF  REPORT  OF  THIS  PAGE  OF  ABSTRACT 

unclassified  unclassified  unclassified 

20.  LIMITATION  OF  ABSTRACT 

NSN  7540-01-280-5500 


Standard  Form  298  (Rev.  2-89) 
Prescribed  by  ANSI  Std.  239-18  298-102 


USAPPC  VI  .00 


United  States  Military  Academy 
West  Point,  New  York  10996 


APPROXIMATING 
PROBABILITY  OF  DETECTION 
IN  THE  JANUS  MODEL 


OPERATIONS  RESEARCH  CENTER 
TECHNICAL  REPORT 
JUNE,  1997 


jyjlC  QUALITY  IHSPECTIBD  i 


Approximating  Probability  of  Detection  in  The 

Janus  Model 


MAJ  Mickey  A.  Sanzotta 
Department  of  Mathematical  Sciences 

and 

MAJ  E.  Todd  Sherrill 
Operations  Research  Center 


U.S.  Military  Academy 
West  Point,  NY  10922 


A  Technical  Report 
of  the 

Operations  Research  Center 
United  States  Military  Academy 


May  1997 


ACKNOWLEDGMENTS 


The  authors  expresses  deep  appreciation  for  the  guidance  and  tutelage  received  from  the 
following  members  of  the  Staff  and  Faculty  of  the  United  States  Military  Academy: 

Dr.  Donald  R.  Barr,  Professor,  Department  of  Systems  Engineering 

MAJ  Dave  Olwell,  Associate  Professor,  Department  of  Mathematical  Sciences 

LTC  Patrick  Driscoll,  Professor,  Department  of  Mathematical  Sciences 

And  to  the  following  for  his  guidance  and  consult  of  the  subject  area: 

Mr.  Paul  West,  Department  of  Systems  Engineering. 


Table  of  Contents 


ACKNOWLEDGMENTS . ii 

I.  PURPOSE . iv 

n.  MODEL  SPECIFICATIONS . 1 

ID.  JANUS  GRAPHICAL  VALIDATION  AND  VERIFICATION . 2 

A.  General . 2 

B.  Plot  Analysis . 3 

C.  Replication  of  Janus  Curves . 3 

IV.  CHOOSING  A  MODEL . 4 

V.  THE  METHOD  OF  LOGISTICS  REGRESSION . 5 

A.  General . 5 

B.  Choosing  a  logistics  regression  model . 6 

C.  Comparison  of  the  Models . 6 

D.  Determining  the  complexity  of  the  models . 8 

E.  The  models . 9 

1.  Ground  Vehicle  Sensors . 9 

2.  Aerial  Sensors . 12 

VI.  CONCLUSIONS . 14 

APPENDIX  A.  DATA  EXTRACTED  FROM  JANUS  V&V  CURVES . 15 

1.  Ground  Vehicle  Sensor  Data  vs.  T80 . 15 

a)  Primary  Sensors . 15 

b)  Secondary  Sensors . 16 


iii 


2.  Aerial  Sensor  Data  vs.  T80 .  16 

a)  Primary  Sensors .  16 

a)  Secondary  Sensors .  17 

APPENDIX  B.  OTHER  MODELS . . 

1 .  Polynomial  Curve  Fitting .  1 8 

2.  Cubic  Spline  Fitting .  lg 

3.  Inverse  Function  Fitting .  19 

REFERENCES .  ™ 


IV 


I.  PURPOSE 


This  report  documents  our  approach  at  developing  a  satisfactory  detection  model 
for  use  in  computing  information  gain  in  the  Janus  Wargame.  Information  gain  measures 
the  Blue  forces’  awareness  of  Red’s  disposition  over  time.  Within  a  time  interval  (t,  t+1) 
the  measure  is  a  distance  measure  between  two  probability  distributions  Pt  and  Pt+i 
respectively.  These  distributions  represent  the  relative  discrete  probability,  from  Blue’s 
perspective,  that  a  Red  vehicle  is  in  a  particular  area  of  the  battlefield.  The  sum  of  the 
discrete  probability  values  over  all  areas  is  1.0  with  those  areas  of  greatest  likelihood 
having  the  larger  values.  Information  is  generated  by  the  actions  of  Blue  sensors.  When 
any  Blue  sensor  scans  an  area  of  the  battlefield  Blue  gains  information  about  the  enemy 
disposition.  The  magnitude  of  this  new  information  is  determined  in  part  by  the 
efficiency  of  the  Blue  sensor.  As  new  information  about  the  location  of  a  Red  vehicle 
becomes  available  to  Blue  we  update  the  probability  distribution  using  theoretical 
formulations  based  upon  Bayes  formula  [4]. 

Though  the  theory  is  simple,  its  implementation  in  the  Janus  model  was  very 
challenging.  The  Bayesian  formulation  mentioned  above  requires  two  parameters  during 
each  time  stage:  1)  a  listing  of  which  areas  of  the  battlefield  (cells)  Blue  sensors  looked 
in  and  2)  the  probability  of  detection  (Pd)  for  the  sensors  that  did  the  respective  scanning. 
Neither  of  these  data  are  directly  available  in  Janus  runs,  nor  can  they  be  deduced  from 
Janus  output  files.  For  example,  the  Janus  algorithms  for  line  of  sight  computations  and 
detection  of  enemy  vehicles  are  only  called  when  two  opposing  vehicles  are  within  some 
threshold  of  proximity  to  each  other.  Since  information  gain  credits  finding  where  the 
enemy  is  not,  we  need  to  know  at  each  time  increment  what  terrain  cells  Blue  sensors 
have  searched  regardless  of  the  presence  or  absence  of  enemy  vehicles.  Likewise,  we 
need  to  know  what  the  Pd  would  have  been  for  each  particular  sensor  /  cell  pair  had  there 
been  an  enemy  vehicle  present  when  the  sensor  searched  the  cell.  For  our  purposes,  a 
sensor  is  considered  to  have  searched  a  cell  if  it  has  unobscured  line  of  sight  between  its 
position  and  the  particular  terrain  cell. 

This  report  explains  an  approach  that  was  used  to  estimate  Pd  values  for  certain 
combinations  of  sensor-target  pairs,  as  a  function  of  sensor-cell  distance.  The 
methodology  involves  fitting  curves  to  PD  values  obtained  from  the  Janus  software  and  is 
illustrated  with  several  example  cases.  Also  discussed  are  alternative  approaches  to 
selection  of  PD  models  and  methods  of  fitting  them  to  the  Janus  detection  values. 


II.  MODEL  SPECIFICATIONS 


The  model  must  produce  Pd  results  in  the  interval  [0,1].  We  prefer  the  simplest 
function  possible  that  adequately  models  detection.  We  chose  to  use  a  function  or  a 
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subroutine  in  our  computations  rather  than  a  table  lookup  method,  since  this  would 
require  us  to  store  data  tables  on  each  sensor/target  pair. 


III.  JANUS  GRAPHICAL  VALIDATION  AND  VERIFICATION 


A.  General. 

The  Janus  software  contains  a  graphical  validation  and  verification  (V&Y) 
section.  We  discovered  in  this  section,  curves  that  provide  a  representation  of  probability 
of  detection  data  for  each  observer-target  pair.  These  graphs  can  be  defined  by  either 
primary  or  secondary  sensors  against  a  stationary  or  moving  enemy  target,  and  can  be 
varied  from  simple  detection  to  actual  identification  of  the  enemy.  We  decided  to  try  to 
replicate  the  information  in  these  curves  with  a  closed  form  function  mapping  the  range 
from  sensor  to  target  into  Pd.  An  example  of  the  type  of  curves  displayed  in  the  Janus 
V&V  section  is  shown  below. 
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Figure  1.  Probability  of  detection  curve  versus  range  (km)  for  a  FistV  w/  thermal 
sights  vs.  a  stationary  (solid  line)  T80. 


2 


B.  Plot  Analysis. 

The  plot  of  a  FISTV  seeking  a  stationary  T80  with  a  Thermal  Sight  is  a 
continuous,  monotonically  decreasing  function  that  begins  at  one  1.0  and  asymptotically 
approaches  zero  as  range  increases.  This  graph  represents  the  probability  of  detection  as 
a  function  of  range,  given  friendly  vehicle/sensor  type  and  enemy  vehicle.  Although  this 
is  a  simplification  of  the  Janus  algorithm,  it  still  represents  the  general  physical  nature  of 
detection  probability.  As  range  increases,  the  likelihood  of  detection  decreases. 


C.  Replication  of  Janus  Curves. 

To  replicate  these  curves,  we  entered  each  graph  for  an  observer-target  pair, 
extracted  ordered  pairs  (range,  PD)  from  the  graph,  and  fit  a  function  to  the  ordered  pairs. 
The  data  we  extracted  from  the  graph  shown  in  Figure  1  is  as  follows: 


Table  1.  Data  extracted  from  Janus  V&V  graphical 
screen,  FistV  with  thermal  sights  vs.  T80. 


■H 

Pd 

.0001 

.9999 

.8 

.9988 

1.6 

.99 

2.4 

.96 

3.2 

.86 

4.0 

.7 

4.8 

.55 

5.7 

.4 

6.4 

.315 

8.0 

.19 

9.6 

.13 

11.2 

.085 

12.8 

.06 

14.4 

.04 

16.0 

.02 

Since  this  is  an  arduous  process,  we  decided  to  limit  the  number  of  observer/target 
pairs  we  would  model.  These  observer-target  pairs  are  actually  an  ordered  triple  of 
vehicle-sensor-target,  since  a  vehicle  can  have  more  than  one  sensor  on  board.  We 
decided  that  we  would  model  the  primary  and  secondary  sensors  on  both  ground  and 
aerial  systems  against  an  enemy  T80  tank.  The  friendly  systems  we  modeled  are  the 
FistV,  Ml,  M2  on  the  ground,  and  the  AH-64  and  OH58D  in  the  air.  We  wanted  to  use 
“killer”  systems  as  well  as  “supporting”  systems  to  diversify  the  PD  curves  and  compare 
their  differences.  In  comparing  the  curves  we  tended  to  favor  those  that  most  accurately 
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modeled  the  killer  systems,  since  there  are  a  greater  number  of  killer  systems  deployed  on 
a  battlefield.  The  table  below  depicts  the  vehicle-sensor-target  triples  we  modeled. 


Table  2.  Vehicle-Sensor-Target  matrix. 


Ground  System  vs.  T80* 

Aerial  System  vs.  T80* 

FistV 

Thermal 

AH-64 

Thermal 

Optical 

Flir 

Ml 

Thermal 

OH-58D 

Thermal 

Optical 

Flir 

M2 

Thermal 

Optical 

|  *T80  is  in  Stationary  Mode 

For  each  of  these  triples,  we  produced  a  table  from  the  corresponding  Janus  V&V 
Screen,  depicting  the  range  and  Pd  for  each  friendly  vehicle-sensor  against  a  stationary 
T80.  The  tabulated  data  can  be  found  in  appendix  A. 

IV.  CHOOSING  A  MODEL 


Since  the  data  defines  Probability  of  Detection  (Pd)  as  a  function  of  Range  for 
each  sensor,  we  were  able  to  fit  a  curve  to  the  data  with  standard  mathematical 
techniques.  The  fitting  procedures  we  attempted  were  polynomial  curve  fitting,  cubic 
spline  interpolation,  fitting  inverse  functions  to  the  data,  conducting  straight  interpolation 
of  tabled  data,  and  performing  logistics  regression  on  the  data.  The  method  we  chose  was 
logistics  regression.  Appendix  B  briefly  explains  the  other  modeling  methods.  We  chose 
logistics  regression  because  the  data  are  produced  by  a  binary  operation  (detection). 
Logistics  regression  also  allows  us  to  perform  multiple  linear  regression  on  the  data  set. 
Thus  one  regression  equation  in  our  algorithm  will  provide  an  estimated  PD  for  all  ground 
and  aerial  systems.  By  using  this  method  we  could  create  a  data  matrix  with  the  range 
and  Pd,  along  with  which  type  of  vehicle  and  sensor  we  were  modeling.  An  abbreviated 
example  of  the  ground  data  matrix  that  we  utilized  in  Excel®,  is  shown  below.  A  similar 
table  was  used  for  the  aerial  data. 
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Table  3.  Abbreviated  multiple  logistics  regression  data  matrix 


Type  Vehicle 

Sensor  Type 

Xranse 

FistV 

Ml 

M2 

Therm 

Optic 

Ydetect 

0.8 

1 

0 

0 

1 

0 

0.9988 

3.2 

1 

0 

0 

1 

0 

0.86 

6.4 

1 

0 

0 

1 

0 

0.315 

0.8 

0 

1 

0 

1 

0 

0.9988 

3.2 

0 

1 

0 

1 

0 

0.805 

6.4 

0 

1 

0 

1 

0 

0.275 

0.8 

0 

0 

1 

1 

0 

0.9988 

3.2 

0 

0 

1 

1 

0 

0.86 

6.4 

0 

0 

1 

1 

0 

0.315 

1.1 

1 

0 

0 

0 

1 

0.9998 

4.4 

1 

0 

0 

0 

1 

0.95 

8.8 

1 

0 

0 

0 

1 

0.45 

1.1 

0 

1 

0 

0 

1 

0.9998 

4.4 

0 

1 

0 

0 

1 

0.999 

8.8 

0 

1 

0 

0 

1 

0.75 

1.1 

0 

0 

1 

0 

1 

0.9998 

4.4 

0 

0 

1 

0 

1 

0.999 

8.8 

0 

0 

1 

0 

1 

0.75 

V.  THE  METHOD  OF  LOGISTICS  REGRESSION 

A.  General. 

Using  a  transformation  function,  T(PD) ,  we  transformed  the  original  dependent 
data  (in  our  case  Pd,  which  is  bounded  by  [0,1])  into  the  Real  line.  We  conducted 
multiple  linear  regression  of  the  transformed  values  upon  independent  variables  such  as 

Range  and  Range^  which  determines  a  linear  prediction  function,  T(PD)  =  g(x)  when  x 
is  the  vector  of  independent  variables.  This  relation  is  re-transformed  through  T~l  to 
obtain  the  prediction  relation  in  the  original  space  [0,1],  PD  =  T  '  (g(x)) .  From  here  on 
we  will  describe  the  dependent  data,  Pd,  as  y  and  the  range  data  as  x. 
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B.  Choosing  a  logistics  regression  model. 

The  most  difficult  part  of  the  process  of  fitting  a  model  to  the  Janus  V&V  curve 
data  was  determining  which  logistic  model  to  utilize.  We  experimented  with  three 
different  types  of  logistic  transformation  functions  shown  as  follows. 

The  first  is  named  the  logit  transformation  [8],  which  is  defined  as: 

e(s(*» 
l+e(g(x))  ’ 

Where  g(x)  =  a  +  (3xx  +  fi2x+---+ p6x  from  our  tabled  data  above,  and: 

a  is  the  y-intercept  in  the  transformed  space. 

/f,  is  the  coefficient  multiplied  by  the  data  in  the  Xrange  Column. 

(3  2  is  the  coefficient  multiplied  by  the  1  or  0  in  the  FistV  Column. 

P6  is  the  coefficient  multiplied  by  the  1  or  0  in  the  Optic  Column. 

The  second  was  the  loglog  transformation  [1],  defined  as: 


7]  00  =  In 


i -y 


,  with  the  associated  inverse  transformation;  nx  (g(x)) 


T2 (y)  =  ln(-ln(l - y)),  with  the  associated  inverse  transformation;  K2(g(x))  =  l-e  . 

And  finally,  the  probit  model  [1],  which  utilizes  the  standard  normal  probability  function, 
defined  by: 


T3  (y)  =  O  1  ,  and  associated  inverse  transformation:  k3  (g(jt))  =  ^ — — 

Again  g(x)  =  a  +  {3tx  +  (32x+---+ (36x  with  corresponding  coefficients  for  the 
loglog  model. 

A  successful  fit  of  one  of  these  models  would  allow  us  to  employ  an  algorithm 
with  two  lines  of  code  that  only  requires  an  identified  vehicle-sensorfground  and  aerial), 
and  a  specified  range  to  produce  an  estimated  PD.  Each  of  these  transformations  were 
modeled  in  Excel®  and  compared  against  one  another. 

C.  Comparison  of  the  Models. 

To  compare  the  three  models,  we  conducted  4th  order  fits  on  all  vehicle/sensor 
pairs  against  the  T80.  This  means  that  we  continued  with  the  same  data  set  as  shown 
above,  but  added  the  variables  of  Xrange2,  Xrange3  and  Xrange4  to  the  model.  The  goal 
was  to  find  one  model  that  adequately  fit  the  ground  based  sensors  and  one  that 
adequately  fit  the  aerial  based  sensor. 


6 


The  multiple  logistics  regression  provided  the  square  of  the  correlation 
coefficient,  R2  value  for  evaluation  of  how  well  the  model  fit  in  the  transformed  space, 
but  not  the  original  data  space  (1,0).  Therefore  we  manually  computed  an  R 2  value 
associated  with  the  original  data  and  the  final  predicted  PD,  from  the  logistics  models 
using  the  formula: 


2 

X0-502  ; 

where  y  is  the  original  PD,  y  is  the  mean  of  the  original  PD,  and  y  is  the  Predicted  PD. 
This  value  of  R 2  was  a  discriminating  factor  in  which  of  the  three  models  we  chose  for 
each  data  set  (ground  and  aerial).  The  R2  values  associated  with  these  4th  order  models 
are  depicted  in  the  table  below,  with  the  most  favorable  R2  value  for  each  model  and 
vehicle  sensor  pair  in  bold  print. 


Table  4.  R2  values  for  4*11  order  multiple  logistic  regression  models,  ground  systems. 


R2  VALUE;  GROUND  SYSTEMS 
VS.  T80 

1 

R2  VALUE;  GROUND  SYSTEMS 

VS.  T80 

_ 

Thermal 

Optical  :! 

FistV 

Ml 

M2 

FistV 

Ml 

M2  1 

| 

.9726 

0.9427 

0.9359 

essesh 

0.4687 

0.8122 

0.7965  1 

.9844 

0.9786 

0.9759 

I18BBW 

0.9246 

0.5866 

-0.508 

MBM 

0.9842 

0.9772 

0.9745 

■ 

lalSSlEIIM 

0.9271 

0.5773 

-0.556 

Table  5.  R2  values  for  4th  order  multiple  logistic  regression  models,  aerial  systems. 


R2  VALUE;  AERIAL  SYSTEMS 

VS.  T80 

1 

R2  VALUE;  AERIAL  SYSTEMS 
VS.  T80 

Thermal 

FLIR  | 

_ 

AH-64 

OH-58D 

AH-64 

OH-58D 

0.98252 

0.97267 

iiifwgy 

0.99839 

0.91197  | 

IliBEfgM 

0.94115 

0.90427 

0.92712 

0.9314 

0.98921 

0.99023 

Probit  4th 

0.9987 

0.9254 

During  this  process  we  discovered  two  factors  affecting  which  model  we  would 

select. 

1)  The  first  was  the  characteristics  of  the  normal  distribution  function.  Although 
the  probit  model  would  work  best  for  the  aerial  sensors,  coding  the  normal  distribution 
function  and  its  inverse  into  an  algorithm  would  require  another  approximation  function, 
since  no  closed  form  solution  exists  for  the  normal  distribution.  The  probit  model  was 
therefore  eliminated  as  an  option  for  our  model.  Thus  we  had  to  replace  the  probit  with 
the  logit  model  ( R2  values  italicized  in  table). 
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2)  The  second  factor  of  concern  was  that  all  of  the  models  adequately  fit  the 
primary  (Thermal)  sensors,  for  each  ground  and  aerial  vehicle,  but  not  the  secondary 
sensors  (Optical  and  FLIR,  respectively)  especially  for  the  ground  optical  sensor.  The 
lack  of  fit  of  these  logistic  regression  models  suggested  that  we  needed  to  utilize  a 
separate  model  for  the  secondary  ground  sensors.  However,  further  research  into  the 
Janus  modeling  system  revealed  that  the  search  algorithm  in  Janus  switches  every  twenty 
(20)  seconds  from  the  primary  to  secondary  scanning  sensors,  until  a  detection  occurs  [5]. 
It  would  be  difficult  to  determine  from  the  Janus  data  base  which  sensor,  primary  or 
secondary,  made  the  initial  detection  of  an  enemy.  Therefore,  we  favored  utilizing  the 
model  which  best  fit  the  primary  sensor.  Further  evaluation  of  the  tabled  data  shows  the 
best  fit  for  the  Ground  vehicle’s  Primary  (Thermal)  Sensor  is  the  Loglog  fit  and  the 
Secondary  (Optical)  is  either  the  Probit,  which  we  eliminated,  or  the  Logit  models.  The 
best  fit  for  the  Aerial  Primary  (Thermal)  Sensors  was  the  Probit  model,  again  eliminated, 
and  the  Probit  and  Loglog  for  the  Secondary  (FLIR)  Sensors.  Since  the  Probit  model  was 
eliminated  for  the  AH-64  Secondary  Sensor,  we  defer  to  the  next  higher  R2  value,  which 
is  the  Logit  model.  Also,  since  the  Logit  model  fit  the  AH-64,  and  the  Loglog  fit  the  OH- 
58D  best,  we  deferred  to  the  AH-64  fit  since  that  vehicle  is  a  killer  system,  and  the  OH- 
58D  is  a  supporting  system. 

Based  on  the  R2  tables  and  the  discussion  above,  we  chose  to  utilize  the  Loglog 
model  for  the  ground  sensors  and  the  Logit  model  for  the  aerial  sensor. 

D.  Limiting  the  complexity  of  the  models. 

We  needed  to  determine  if  we  could  use  a  less  complex  model,  i.e.  a  3rd  order  or 
even  2nd  order  model.  We  desire  to  use  the  least  computationally  expensive  algorithm 
possible.  We  wanted  to  reduce  the  complexity  of  our  algorithm  without  losing  an 
excessive  amount  of  accuracy.  We  took  the  R2  values  of  the  4th  order  models  and 
compared  them  to  the  R2  values  of  the  3rd  order  model,  and  likewise,  3rd  to  2nd  order. 
From  that  evaluation  we  determined  the  percent  gain  of  using  a  more  complex  model.  If 
the  percent  gain  did  not  appear  significant,  say  less  than  one  percent,  we  decided  not  to 
use  the  more  complex  model.  The  comparisons  of  the  R2  values  are  shown  below. 


Table  6.  Comparison  of  R2  values  for  loglog  multiple  logistic  regression  model,  ground. 


R2  Values  for  LogLog  Model,  Ground  Vehicle  Sensors  l 

(Thermal) 

2nd  Order 

3rd  Order 

Delta2  -  3 

%  Gain 

4th  Order 

Delta3  -  4 

%  Gain 

FistV 

0.984421 

0.98508 

0.000659 

0.067% 

0.98442 

-0.00066 

-0.067% 

Ml 

0.97867 

0.97824 

-0.00043 

-0.044% 

0.97867 

0.00043 

0.044% 

M2 

0.97596 

0.97867 

0.00271 

0.278% 

0.97596 

-0.00271 

-0.277% 
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KBBB 

odel,  Aerial  Sensors  (Thermal) 

- 1 

BflKoBa 

Delta2  -  3 

%Gain 

4th  Order 

Delta3  -  4 

KfciHII 

AH-64 

0.929474 

0.981245 

0.051772 

5.570% 

0.981362 

0.000117 

0.012% 

OH-58D 

0.953021 

0.965042 

0.012021 

1.261% 

0.972673 

0.007631 

0.791% 

These  tables  show  that  the  ground  vehicle  sensors  can  be  adequately  modeled 
using  a  2  order  Loglog  model.  This  is  because  the  greatest  percent  gain  in  accuracy  we 
can  achieve  by  using  a  3rd  order  model  is  0.278%,  which,  for  our  purposes,  was  not  large 
enough  to  warrant  increased  complexity.  Also  we  see  that  a  3rd  order  Logit  model  is 
accurate  enough  because  the  greatest  percent  gain  in  accuracy  we  can  achieve  by  using  a 
4*  order  model  is  0.791%,  which  again,  is  not  large  enough  to  increase  the  complexity  of 
our  algorithm.  Thus  the  models  we  settled  upon  are  the  2nd  order  Loglog  multiple  logistic 
regression  model  for  the  ground  vehicle  sensors  and  the  3rd  order  Logit  logistic  multiple 
regression  model  for  the  aerial  sensors. 


E.  The  models. 

1.  Ground  Vehicle  Sensors. 

The  2nd  order  Loglog  regression  model  for  ground  vehicle  sensors: 

T{y)  =  ln(— ln(l  —  y)) ,  with  the  associated  inverse  transformation;  7r(g(x))  =  1  —  e~^g(x^ , 

Where  g(x)  =  a  +  P,x,  +  p2xf  +  P3x3  +P4x4  +Psx5+ p6x6+ P7x7 ,  and: 
a  is  the  y-intercept  in  the  transformed  space. 

Pt  is  the  coefficient  multiplied  by  the  data  in  the  Xrange  Column. 

P2  is  the  coefficient  multiplied  by  the  data  in  the  Xrange2  Column. 

P3  is  the  coefficient  multiplied  by  the  1  or  0  in  the  FistV  Column. 

P 4  is  the  coefficient  multiplied  by  the  1  or  0  in  the  Ml  Column. 

P5  is  the  coefficient  multiplied  by  the  1  or  0  in  M2  Column. 

P6  is  the  coefficient  multiplied  by  the  1  or  0  in  the  Thermal  Column. 

Pi  is  the  coefficient  multiplied  by  the  1  or  0  in  Optical  Column. 

The  final  equation  for  g(x)  will  look  like: 

g(x)  =  1 6.244877  -  0.45 1 966x,  +0.006601 Ax?  -0.313104x3  -0.2118893x4  +0x5 
-  14.058842x6  -  12.466667x7 
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which  will  be  coded  into  the  detection  algorithm  along  with  7r(g(x))  =  l-e  e(s(X>> ,  which 
will  predict  our  probability  of  detection,  Pd,  for  ground  vehicle  sensors. 

The  three  graphs  of  our  predicted  curves  and  the  original  data  curves  for  each 
ground  sensor  (FistV,  Ml,  M2)  are  shown  pictorially  below. 


FistV  vs.  T80  (Therm)  Quad  Fit  (log-log) 

1 

0.9 
0.8 
0.7 

|  0.6 
®  0.5 
2  0.4 

Pn 

0.3 
0.2 
0.1 
0 

0  5  10  15 

Range 

Figure  2.  Graphic  comparison  of  Loglog  quadratic  fit  to  original  data,  (FistV  (therm)  vs.  T80). 


LogLog2 

Ydetect 
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M2  vs.  T80  (Therm)  Quad  Fit  (log-log) 


Range 


— LogLog2 
— ■—  Ydetect 


Figure  4.  Graphic  comparison  of  Loglog  quadratic  fit  to  original  data,  (M2  (therm)  vs.  T80). 


2.  Aerial  Sensors. 

The  3rd  order  Logit  regression  model  for  aerial  sensors: 

e{gM) 

l  +  e(*w)  ’ 

Where  g(x)  =  a  +  px x,  +  P2x]  +  p3x3  +  PAxA  +  p5x5  +  p6x6  +P7x7,  and: 
a  is  the  ^-intercept  in  the  transformed  space. 

P{  is  the  coefficient  multiplied  by  the  data  in  the  Xrange  Column. 

P2  is  the  coefficient  multiplied  by  the  data  in  the  Xrange2  Column. 

I 83  is  the  coefficient  multiplied  by  the  1  or  0  in  the  Xrange3  Column. 

P4  is  the  coefficient  multiplied  by  the  1  or  0  in  the  AH-64  Column. 

P5  is  the  coefficient  multiplied  by  the  1  or  0  in  OH-58D  Column 

P6  is  the  coefficient  multiplied  by  the  1  or  0  in  the  Thermal  Column. 

P7  is  the  coefficient  multiplied  by  the  1  or  0  in  Flir  Column. 


T(y)  =  In 


.1  -y 


,  with  the  associated  inverse  transformation;  n(g(x)) 
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The  final  equation  for  g(*)  will  look  like: 


g(x)  =  866.98 11 62 -2.25431  lx,  +  0.136956a:,2  -0.002840*?  -  788.398867*4 
-789.333333*5  -67.706670;t6  -66.92857  lx, 

e(sM) 

which  will  be  coded  into  the  detection  algorithm  along  with  7r(g(*))  =  - — j-— ,  which 


will  predict  our  probability  of  detection,  Pd,  for  aerial  sensors. 


The  two  graphs  of  our  predicted  curves  and  the  original  data  curves  for  each  aerial 
sensor  (AH-64  and  OH-58D)  are  shown  pictorially  below. 


Figure  5.  Graphic  comparison  of  Logit  cubic  fit  to  original  data,  (AH-64  (therm)  vs.  T80). 
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OH58D  vs.  T80  (Therm)  Cubic  Fit  (Logit) 


i 


0.9 

0.8 

0.7 

pi 

© 

0.6 

© 

Q 

0.5 

© 

U 

0.4 

CL, 

0.3 

0.2 

0.1 

0 

0  5  10  15  20  25 

Range 

Figure  6.  Graphic  comparison  of  Logit  cubic  fit  to  original  data,  (OH-58D  (therm)  vs.  T80). 


VI.  CONCLUSIONS 


The  data  extracted  from  the  Janus  V&V  screens  is  an  estimate  of  what  the  screens 
depict.  These  screens  simplify  the  actual  PD  algorithm  in  Janus  by  reducing  the  PD  to  a 
function  of  range,  given  a  friendly  sensor  and  an  enemy  target.  We  have  simplified  the 
model  further  by  singularly  evaluating  a  PD  curve  of  friendly  vehicles  against  a  T80  tank. 
We  did  this  because  we  would  have  no  idea  what  type  of  enemy  the  friendly  vehicle 
would  encounter,  thus  we  assumed  that  all  friendly  forces  would  look  for  the  most 
dangerous  ground  force  enemy,  a  T80  tank. 

The  logistics  regression  models  provide  an  accurate  model  of  the  physical  nature 
of  detection.  The  chosen  Loglog  and  Logit  models  provide  an  adequate  fit  to  the  data 
provided  by  the  Janus  V&V  curves.  Multiple  logistics  regression  is  a  powerful  model  to 
use  in  analyzing  the  data  because  1)  it  provides  output  in  the  [0,1]  space,  and  2)  we  can 
use  one  model  that  fits  the  three  ground  systems  and  one  model  that  fits  the  two  aerial 
systems.  This  can  be  accomplished  with  reasonably  conservative  computation  time  and 
data  storage. 
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APPENDIX  A.  DATA  EXTRACTED  FROM  JANUS  V&V  CURVES. 


The  following  tables  represent  the  extracted  data  of  ordered  pairs  (range,  Pd)  from 
the  Janus  Validation  and  Verification  graphs  produced  for  an  observer- target  pair  for  each 
type  of  sensor.  See  figure  1  and  Table  1. 


The  heading  consists  of  type  of  friendly  vehicle  with  type  of  sensor  against  a 
stationary  enemy  T80.  The  left  hand  column  is  the  sensor’s  range  and  the  right  hand 
column  is  the  probability  of  detection  PD. 

1.  Ground  Vehicle  Sensor  Data  vs.  T80. 


a)  Primary  Sensors. 


1 

3? 

H 

80  (Therm) 

Xrange 

Ydetect 

0.8 

0.9988 

1.6 

0.99 

2.4 

0.96 

3.2 

0.86 

4.0 

0.7 

i  4.8 

0.55 

5.7 

0.4 

6.4 

0.315 

8.0 

0.19 

9.6 

0.12 

11.2 

0.085 

12.8 

0.06 

|M1  vs.  T80  (Therm) 

Ydetect 

0.8 

0.9988 

1.6 

0.99 

2.4 

0.94 

3.2 

0.805 

3.6 

0.7 

4.1 

0.6 

4.8 

0.485 

5.4 

0.4 

6.4 

0.275 

7.3 

0.2 

8.0 

0.16 

9.6 

0.1 

11.2 

0.075 

12.8 

0.045 

14.4 

0.03 

M2  vs.  T8( 

(Therm) 

Xrange 

Ydetect 

0.8 

0.9998 

1.6 

0.999 

3.0 

0.9 

3.2 

0.86 

4.0 

0.7 

4.8 

0.555 

5.8 

0.4 

6.4 

0.315 

8.0 

0.19 

9.6 

0.115 

11.2 

0.08 

12.8 

0.055 

14.4 

0.04 
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b)  Secondary  Sensors. 


FistV  vs.  T! 

80(Optical 

Xrange 

Ydetect 

1.1 

0.9998 

2.2 

0.9997 

3.3 

0.999 

4.4 

0.95 

5.5 

0.93 

6.6 

0.8 

7.7 

0.62 

8.8 

0.45 

9.9 

0.31 

M2  vs.  T80  (Optical) 

Xrange 

Ydetect 

1.1 

0.9998 

2.2 

0.9997 

3.3 

0.9996 

4.4 

0.9995 

5.5 

0.999 

6.6 

0.98 

7.7 

0.94 

8.8 

0.86 

9.9 

0.75 

mm m 

Xrange 

Ydetect 

1.1 

0.9998 

2.2 

0.9997 

3.3 

0.9996 

4.4 

0.999 

5.5 

0.995 

6.6 

0.95 

7.7 

0.875 

8.8 

0.75 

9.9 

0.61 

2.  Aerial  Sensor  Data  vs.  T80. 


a)  Primary  Sensors. 


AH-64  vs.  T80  (Therm)  | 

Xrange 

Ydetect 

1.1 

0.9998 

2.2 

0.999 

3.3 

0.98 

4.4 

0.915 

5.5 

0.799 

6.6 

0.65 

7.7 

0.515 

8.8 

0.4 

9.9 

0.31 

11 

0.255 

12.6 

0.185 

14.7 

0.115 

16.8 

0.085 

18.9 

0.06 

21 

0.045 

23.4 

0.035 

|OH-58D  vs.  T80  (Therm)  I 

Xrange 

Ydetect 

1.2 

0.9998 

2.4 

0.995 

3.6 

0.95 

4.8 

0.845 

6 

0.69 

6.6 

0.6 

7.2 

0.52 

8.4 

0.399 

9.6 

0.299 

10.8 

0.22 

12 

0.18 

13 

0.145 

15.6 

0.09 

18.2 

0.06 

20.8 

0.04 

23.4 

0.03 
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a)  Secondary  Sensors. 


|AH-64  vs.  1 

r80  (Flir) 

Ydetect 

0.5 

0.9999 

1 

0.9998 

1.5 

0.9998 

2 

0.9997 

2.5 

0.9997 

3 

0.9996 

3.5 

0.  9996 

4 

0.999 

4.5 

0.98 

|OH-58D  vs.  T80  (Flir)  \ 

Ydetect 

0.5 

0.9998 

l 

0.9998 

1.5 

0.9997 

2 

0.9997 

2.5 

0.9996 

3 

0.999 

3.5 

0.975 

4 

0.85 

4.1 

0.8 

4.3 

0.7 

4.5 

0.595 

APPENDIX  B.  OTHER  MODELS. 


This  appendix  briefly  explains  other  modeling  techniques  that  we  investigated  for 
use  in  modeling  probability  of  detection.  For  various  reasons  described  below,  these 
methods  were  not  accepted  as  viable  solutions  to  our  problem. 


1.  Polynomial  Curve  Fitting. 

We  utilized  least  squares  regression  on  the  data  to  determine  predicted  values  of 
Pd  at  the  given  ranges.  The  data  looked  as  if  a  cubic  or  fourth  order  polynomial  would  fit 
best.  MATLAB®  was  utilized  to  perform  the  polynomial  fitting.  The  cubic  polynomial 
fit  yielded  the  polynomial: 

/3(x)  =.0003x3-.0004x2—  .1275x  +  1  with  an  R2  =8664 

the  fourth  order  polynomial  fit  yielded: 

f4(x)  =  -0002x4+.0052x3-.0491x2-.0299x  +  1  with  an  R2  =9886 

The  equation  f4(x)  provides  an  adequate  fit  to  the  data.  However,  there  are  areas  of 
concern  in  using  this  function.  The  first  concern  is  that  just  near  zero,  the  value  of  the 
function  f4(x )  output  would  be  slightly  greater  than  one.  Secondly,  the  function  f4(x)  is 
not  monotonically  decreasing  after  10  kilometers  in  range.  It  is  unrealistic  for  the  PD  to 
begin  to  increase  as  range  increases.  Although  these  situations  can  easily  be  accounted 
for  in  an  algorithm  with  ‘conditional  if  statements,  we  did  not  want  to  have  to  put  a 
condition  on  each  of  the  possible  polynomial  models  used  in  our  algorithm. 


2.  Cubic  Spline  Fitting. 

The  second  method  we  used  to  model  the  PD  was  cubic  spline  fitting.  This  method 
interpolates  between  data  points  using  cubic  spline  fits.  This  turned  out  to  be  a  powerful 
tool  since  the  spline  fit  passes  through  each  of  the  (x,  y)  data  points.  Yet  again,  the 
phenomena  existed  such  that  around  1.5  to  2  kilometers  in  range,  the  modeled  value  for 
Pd  is  greater  than  one.  The  other  problem  with  the  cubic  spline  model  fit  is  that  it  is  not  a 
function,  but  a  set  of  data  points  represented  in  a  table.  This  type  of  output  would  require 
us  to  store  a  data  set  for  each  sensor  /  vehicle  pair,  and  then  enter  into  the  data  table  with 
a  search  algorithm  to  determine  the  PD. 
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3.  Inverse  Function  Fitting. 


Similar  to  the  above  methods,  we  attempted  to  fit  inverse  functions  to  the  data.  These 
functions  provided  S  shaped  curves  that  are  bound  by  [0, 1]  and  closely  resembled  the 
data.  However,  these  functions  had  to  be  physically  manipulated  in  order  to  provide  the 
shape  of  each  different  sensor-target  pair.  As  an  example,  we  fit  the  Probability  of 
Detection  of  an  Ml  vs.  a  T80  using  a  thermal  sight  with  the  function: 


//(*)  = 


15 

(x2  +15)  ’ 


where  the  number  15  was  selected  off  the  Janus  graph  as  a  range  value  where  Pd  is  near 
zero.  This  function  fit  the  data  quite  well,  with  an  R 2  =.9244 .  This  method  was  not 
selected  because  a  trial  and  error  method  is  required  to  determine  which  inverse  function 
accurately  fits  the  data. 
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